Teaching Likelihood based Statistical Inference with Mathematica using SIP

Introduction to Statistical Inference Package SIP

Esa Uusipaikka

Statistical Inference Software Inc./
University of Turku

Statistical inference and SIP

Observations

Statistical inference is always based on observations from the phenomenon under consideration. The set of observations is the first necessary component of statistical evidence on which the inference relies.

Statistical model

The second necessary component is a statistical model. The statistical model is based on the assumption that the observations contain random variation, that is, can be considered to have arisen from some probability distribution.

Parameter function of interest

Statistical inference concerns some characteristic or characteristics of the phenomena from which the observations have arisen. The characteristics of the phenomena under consideration are some functions of the unknown parameter of the statistical model and are called the parameter functions of interest.

Statistical inference and SIP (con't)

Measures of uncertainties

Statistical inference consists of statements concerning the unknown value(s) of the interest function(s). Statistical inference differs from other possible modes of inferences in that it always gives measures of uncertainties of the statements made.

Profile likelihood based intervals and likelihood ratio tests

The SIP-package has been developed mainly to assist applied statisticians to make inferences on a real valued parameter function of interest. The package contains functions for calculation of the so-called profile likelihood based intervals and their uncertainties for the unknown value of the real-valued interest function. It includes also functions to make so-called likelihood ratio tests concerning some given hypothesis about the parameters of the statistical model.

Example

This initializes the SIP-package.

<<StatisticalInference`

Observations

Let the observations consist the following 10 real numbers.

(Subscript[y, obs]) = {4.74, 3.14, 4.27, 0.73, 1.78, 0.28, 1.04, 3.77, 10.79, 
          6.96};

Statistical model

Assume that the observations are a random sample of size 10 from gamma distribution with unknown shape ν and mean μ.

ℳ = 
      SamplingModel[GammaModel[ν, μ], 10]

Example (con't)

Confidence interval

If variance of the gamma distribution (((μ^2))/ν) is of interest, this gives 95% profile likelihood based confidence interval to the unknown variance.

ProfileInterval[ℳ,(Subscript[y, obs]),(((μ^2))/ν),{ν, μ}]//ConfidenceLimits

Maximum likelihood estimate

Because the determination of profile likelihood intervals and significance of likelihood ratio tests requires maximum likelihood estimation SIP-package contains a function for determination of this maximum likelihood estimate.

MLEFit[ℳ, (Subscript[y, obs])] // MLParameterEstimate

Statistical distributions in SIP

Discrete models

Statistical distributions in SIP (con't)

Continuous models

Statistical models

Statistical models describe the random variation in data. In SIP-package there is a collection functions for constructing statistical models. In these functions the building blocks from which the statistical model is constructed can be statistical distributions or statistical models. This recursive way of defining statistical models allows a huge variety of statistical models.

This defines a statistical model for two independent observations from binomial distributions with 10 and 20 trials, respectively, and possibly different unknown success probabilities ω and θ.

ℳ = 
      IndependenceModel[{BinomialModel[10, ω], 
          BinomialModel[20, θ]}]

In SIP-package also response is a list of numbers, but can have a more complicated structure. This shows the structures of the response and the parameter of the statistical model ℳ.

ShowResponse[ℳ]

ShowParameter[ℳ]

Statistical models  (con't)

This tests the hypothesis that the distributions have common unknown success probability η given 2 and 8 successes, respectively.

LRTest[ℳ, {2, 8}, η, {η, η}] // 
      ObservedSignificanceLevel

Statistical models in SIP

Basic models

Regression models

Models for stochastic processes

Statistical models in SIP  (con't)

Hierarchical models

Properties of statistical distributions and models

SIP-package contains a collection of functions to calculate various properties of statistical distributions and models. Most of these allow symbolic expressions for responses and parameters.

Density function

This gives the density function of the normal distribution in symbolic form.

DensityFunction[NormalModel[μ, σ], 
      y, {μ, σ}]

Properties of statistical distributions and models  (con't)

Log-likelihood function

This gives the log-likelihood function of the statistical model for a sample of size 5 from the Student t-distribution with degrees of freedom, location, and scale unknown.

LogLikelihoodFunction[
      SamplingModel[StudentTModel[ν, μ, σ], 5], {y1, y2, y3, 
        y4, y5}, {ν, μ, σ}]

Properties of statistical distributions and models  (con't)

Random sample

This generates a random sample of size 100 from the Poisson distribution with mean 10.4.

RandomSampleAtParameter[PoissonModel[μ], 100, 10.4]

Properties of statistical distributions and models  (con't)

Version 1.0 of SIP-package

Downloadable from

http://www.wolfram.com/products/applications/sip/

Technical support from

support@statisticalinference.com

More information from

http://www.statisticalinference.com

Developed by

Esa Uusipaikka
	  Professor of Statistics
	
	  Head of Depertment of Statistics	
	  University of Turku
	  FI-20014 Turun yliopisto
	  Finland	
	
	  Phone:               +358-2-333 5436
	  Email:                esa.uusipaikka@utu.fi
	  WWW:               http://users.utu.fi/esauusi/esauusi.htm